Web Survey Bibliography
Web surveys have been a commonly used method for collecting and analyzing data about social and psychological phenomena in online communities. Nevertheless, as yet, little systematic methodological research has been done about administrating and conducting surveys in these online domains. This is somewhat surprising due to the methodological advances of other research techniques in the field of online community research such as virtual ethnography, data mining and social network analysis and to the fact that internet surveys were among the first techniques used in online community research (Jones 1998, 1999).Quality of web surveys in online communities is commonly affected by two types of survey errors (Andrews et al. 2003): non-coverage and unit nonresponse error. Although both errors can be interrelated, on one hand, the noncoverage error mainly applies to the issue of defining a valid unit of analysis (on theoretical and empirical level). Generally online communities require from individuals a kind of formal registration in order to participate in community activities (e.g., in order to submit a message to a forum thread one must log in with their email to an online community). However, there are also online communities that allow individuals to participate without being necessarily registered (i.e., anonymous users). In such cases, researchers often opt for intercept surveys and/or unrestricted self-selected surveys (e.g., by posting an invitation message to a forum discussion thread or publish a banner on the website) (Fricker 2010) failing to give any chance of sample selection to some users of the analyzed online community. On the other hand, the unit nonresponse error generally pertains to list-based web surveys (Lozar Manfreda et al. 2002). In such cases a sampling frame of online community members is available (e.g., a list of all members of an online community), but due to various reasons some sample units refuse to answer the survey. As in general population surveys also in online community surveys unit nonresponse can lead to bias. For instance, there is some empirical evidence that a larger percentage of high-frequency online community participants takes part in surveys compared to low-frequency participants and that high-frequency participants reply quicker to survey invitations than low-frequency participants (Andrews et al. 2003).The purpose of this paper is to present a research design of a study aiming at evaluating the effect of survey invitation characteristics and response reluctance on unit nonresponse in list-based surveys of web forum participants. On one hand, we are interested in response-inducing techniques (Dillman et al. 2007). Up to now the literature on list-based web surveys has been generally focused on the topic of personalization of email invitations, their subject lines and graphical design. The focus of this study is on the content of email invitations. Namely, we will attempt to gain an insight into how response rates are associated with request for help, authority, and sense of community. The decision to study a combination of these three elements was theoretically informed by research on online communities, showing that exchange of social support, belonging and norms are among the most salient building blocks of their development and sustainability (Preece 2000, Kraut et al. 2012). Considering the leverage-salience theory (Groves et al. 2000) and the fact that the these elements are not appealing for all online community members to the same extent, putting forward different combinations of the three elements in invitations might therefore result in different response rates. In addition, we would like to explore if and how the three response-inducing techniques are associated with the unit nonresponse bias. In order to address these questions an experiment with a full factorial between-subjects design with eight experimental groups was planned to be implemented. On the other hand, the research design was developed to explore whether there are differences between early and late respondents as well as if sending reminders can reduce the potential bias associated with unit non-response error. The advantage of list-based web surveys in web forums is that nonresponse bias can be analyzed with the record-linkage technique (Porter and Whitcomb 2005) since both respondents and nonrespondents to the survey can be linked to database records of registered web forum users (which generally includes information about frequency of participation, number of posted messages, year of registration, etc.). The presentation will end with an open discussion of potential avenues for optimization of the proposed research design.
Workshop Homepage (abstract)
Web survey bibliography (272)
- Usability Testing for Survey Research; 2017; Geisen, E.; Romano Bergstrom, J. C.
- A Case Study on Evaluating the Relevance of Some Rules for Writing Requirements through an Online Survey...; 2017; Warnier, M.; Condamines, A.
- Web Health Monitoring Survey: A New Approach to Enhance the Effectiveness of Telemedicine Systems ; 2016; Romano, M. F.; Sardella, M. V.; Alboni, F.
- Design and test of a web-survey for collecting observer’s ratings on dairy goats’ behavioural...; 2016; Vieira, A.; Oliveira, M. D.; Nunes, T.; Stilwell, G.
- A look at the unique data-gathering process behind the Harvard Impact Study; 2016; Vitale, J.
- Analyzing Cognitive Burden of Survey Questions with Paradata: A Web Survey Experiment; 2016; Hoehne, J. K.; Schlosser, S.; Krebs, D.
- Gamification of Online Surveys: Design Process, Case Study, and Evaluation; 2015; Harms, J.; Biegler, S.; Wimmer, C.; Kappel, K.; Grechenig, T.
- Finding Item Nonresponse Patterns: Three Internet Survey Experiments Into the Effects of Nonresponse...; 2015; Van De Maat, J.
- The Effects of Adding a Mobile-Compatible Design to the American Life Panel; 2015; Toepoel, V.; Lugtig, P. J.; Amin, A.
- Technology and Reporting of Daily Activities – Considerations for Analysis of Behaviours in Mixed...; 2015; Fisher, K.; Gershuny, J.
- Cheating in web surveys. Evidence from a split-ballot repeated experiment on knowledge questions on...; 2015; Ladini, R.; Vezzoni, C.
- Unplanned use of mobile devices in a probabilistic online panel survey: Patterns of use and implications...; 2015; Poggio, T.; Bosnjak, M.; Bandilla, W.; Weyandt, K.
- The importance of scale direction between different modes; 2015; Agalioti-sgompou, V.
- Examining the Impact of Mobile First and Responsive Web Design on Desktop and Mobile Respondents; 2015; Tharp, D.
- Boosting Probability-Based Web Survey Response Rates via Nonresponse Follow-Up; 2015; Chew, K.; Fontes, A.; Lavrakas, P. J.
- Cognitive Testing of Survey Translations: Does Respondent Language Proficiency Matter?; 2015; Schoua-Glusberg, A.; Park, H.; Meyer, M.; Goerman, P. L.; Sha, M.
- Questionnaire length and breakoffs in web surveys: a meta study; 2014; Vehovar, V., Cehovin, G.
- Inside the Turk Understanding Mechanical Turk as a Participant Pool; 2014; Paolacci, G., Chandler, J.
- Social Media and Online Survey: Tools for Knowledge Management in Health Research ; 2014; Merolli, M., Sanchez, F. J. M., Gray, K.
- Development and validation of a single- item scale for the relative assessment of physical attractiveness...; 2013; Lutz, J.; Kemper, C. J.; Beierlein, C.; etc.
- A standard with quality indicators for web panel surveys: a Swedish example; 2013; Nyfjaell, M.
- Developing a New Mixed-Mode Methodology For a Provincial Park Camper Survey in British Columbia; 2013; Dyck, B. W.
- Scientific impact of the MESS Project: A brief overview; 2013; Das, M.
- Using the iPad as a Prize-Based Incentive to Boost Response Rates: A Case Study at Brigham Young University...; 2013; McClendon, R., Olsen, D.
- Using Qualitative and Quantitative Testing to Improve Hispanic Response Rates for Online Surveys; 2013; Pens, Y., Gentry, R. J.
- The ONS Beyond 2011 Programme & possible implications for social surveys; 2013; Morris, L.
- Survey Research; 2013; Abbott, M. L., McKinney, J.
- The effect of short formative diagnostic web quizzes with minimal feedback; 2013; Baelter, O., Enstroem, E., Klingenberg, B.
- Web CATI (Part of NatCen’s Multi-Mode Approach) ; 2012; Damestani, P., Agur, M.
- What is Online Research?: Using the Internet for Social Science Research; 2012; Hooley, H., Wellens, J., Marriott, J.
- WebSM Study: Survey software features overview ; 2012; Vehovar, V., Cehovin, G., Kavcic, L., Lenar, J.
- What Survey Modes are Most Effective in Eliciting Self-Reports of Criminal or Delinquent Behavior?; 2012; Kleck, G., Roberts, K.
- Assessing Cross-National Equivalence of Measures of Xenophobia: Evidence from Probing in Web Surveys; 2012; Behr, D., Braun, M., Kaczmirek, L.
- Adaptive web sampling in ecology; 2012; Thompson, S. K.
- Online Data Collection in the Agro-Food Sector; 2012; Biffignandi, S., Artaz, R.
- Psychometric properties of an internet administered version of the Marlowe-Crowne Social Desirability...; 2012; Vesteinsdottir, V., Reips, U.-D., Joinson, A. N., Porsdottir, F.
- Research design for studying online communities with web surveys; 2012; Petrovcic, A., Petric, G., Lozar Manfreda, K.
- Case study: Respondent perspective on survey response; 2012; Jarrett, C.
- Presidential Elections in Iceland 2012 – Did online panel surveys give false hope to new candidates...; 2012; Jonsdottir, G. A., Dofradottir, A. G., Bjornsdottir, A. E.
- Internet Mobility Survey Sampling Biases in Measuring Frequency of Use of Transport Modes ; 2012; Diana, M.
- Qualitatively Speaking: Mobile qualitative finally hits its stride; 2012; Bryson, J.
- Using Collaborative Web Technology to Construct the Health Information National Trends Survey; 2012; Moser, R. P., Beckjord, E. B., Finney Rutten, L. J., Blake, K., Hesse, B. W.
- A Shot in the Dark: Measurement Influence on Likelihood to Vaccination; 2012; Higgins, W. B., Thomas, R. K.
- Using Online Panels for National Surveys of Low Incidence Populations: Findings from the CDC Influenza...; 2012; Boyle, J., Ball, S., Ding, H., Srinath, K. P., Euler, G.
- Drop-Off Point for Undergraduate Students on a Web-based Alcohol and Tobacco Use Questionnaire; 2012; Mitra, A.
- An Examination of the 2010 Census Be Counted Program and Its Effects on Census Coverage and Duplication...; 2012; Jackson, G. I., Wechter, K. M.
- The Detection and Effects of Data From Potentially Ineligible Participants in Online Survey Research...; 2012; Grey, J.
- Internet Mobility Survey Sampling Biases in Measuring Frequency of Use of Different Transport Modes; 2012; Diana, M.
- Continuous large-scale volunteer web-surveys: The experience of Lohnspiegel and WageIndicator; 2012; Oez, F.
- FamilyVote – Conducting online surveys with children and families; 2012; Geissler, H., Peeters, H.